Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3.9 fixups #332

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Conversation

asheplyakov
Copy link

As of now CHIRP fails to start with Python 3.9:

  • Complains about missing importlib_resources module (although importlib is available as a part of the standard library)
  • Application icon and stock configs can't be found (since chirp.share is not a proper python package)

With this patchset I can use CHIRP with Python 3.9.

I get the following exception on start:

$ python3 chirpwx.py
Traceback (most recent call last):
File "/home/asheplyakov/work/radio/chirp/chirpwx.py", line 8, in
<module>
    sys.exit(chirpmain())
File "/home/asheplyakov/work/radio/chirp/chirp/wxui/__init__.py", line
41, in chirpmain
    from chirp.wxui import main
File "/home/asheplyakov/work/radio/chirp/chirp/wxui/main.py", line 28,
in <module>
    import importlib_resources
ModuleNotFoundError: No module named 'importlib_resources'

$ python3 --version
Python 3.9.6

However importlib is available in Python 3.9 (as a part of
the standard library). To avoid the problem this patch removes
the version check, and checks for
importlib_resources/importlib.resources modules instead.
I get the following error on startup:

$ python3 chirpwx.py

Traceback (most recent call last):
  File "/home/asheplyakov/work/radio/chirp/chirpwx.py", line 8, in <module>
    sys.exit(chirpmain())
  File "/home/asheplyakov/work/radio/chirp/chirp/wxui/__init__.py", line 94, in chirpmain
    mainwindow = main.ChirpMain(None, title='CHIRP')
  File "/home/asheplyakov/work/radio/chirp/chirp/wxui/main.py", line 307, in __init__
    self.set_icon()
  File "/home/asheplyakov/work/radio/chirp/chirp/wxui/main.py", line 342, in set_icon
    importlib_resources.files('chirp.share')
  File "/usr/lib64/python3.9/importlib/resources.py", line 147, in files
    return _common.from_package(_get_package(package))
  File "/usr/lib64/python3.9/importlib/_common.py", line 14, in from_package
    return fallback_resources(package.__spec__)
  File "/usr/lib64/python3.9/importlib/_common.py", line 18, in fallback_resources
    package_directory = pathlib.Path(spec.origin).parent
  File "/usr/lib64/python3.9/pathlib.py", line 1072, in __new__
    self = cls._from_parts(args, init=False)
  File "/usr/lib64/python3.9/pathlib.py", line 697, in _from_parts
    drv, root, parts = self._parse_args(args)
  File "/usr/lib64/python3.9/pathlib.py", line 681, in _parse_args
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

The problem is that 'chirp.share' is not a proper python package,
therefore importlib fails to locate its resources.
To avoid the problem this patch adds empty __init__.py files to
chirp/share and chirp/stock_configs directories.

As a side effect setup.py installs (the content of) these directories
(which makes packagers more happy).
@asheplyakov
Copy link
Author

asheplyakov commented Dec 29, 2022 via email

@kk7ds
Copy link
Owner

kk7ds commented Dec 29, 2022

They're good enough for the bundler to pick up - I didn't really think they had to be proper python packages to be considered resources, but I dunno. Maybe @vishwin has some context here, but I can poke with python3.9 a little later. I thought that the thing we specifically needed was only in 3.10...

@vishwin
Copy link
Contributor

vishwin commented Dec 29, 2022

The new-style API was introduced in Python 3.10, and to simplify maintenance (especially when Python 3.11 formally marked the old-style but imo more concise API deprecated), the package is used for Python < 3.10.

@vishwin
Copy link
Contributor

vishwin commented Dec 29, 2022

As for package detection itself, setuptools does (or at least did) consider chirp.share a package implicitly, but of course explicit is always better than implicit.

@kk7ds
Copy link
Owner

kk7ds commented Dec 29, 2022

@asheplyakov Is it possible that just the __init__.py additions make it work for you? Like @vishwin it surely seems like 3.9 should require importlib_resources to work properly. Have you verified that your pip install command is actually installing that lib and that it's importable via the python that you're using to run chirp?

@kk7ds
Copy link
Owner

kk7ds commented Dec 29, 2022

On a clean python 3.9 docker image, python3 setup.py install installs importlib-resources as a dep as expected, and I can launch chirp from the installed directory just fine and it runs, finds the stock stuff, icons, etc, which I can confirm is installed to the proper location.

So, I'm not sure what the problem is, but I don't think it's specific to 3.9.

@goldstar611
Copy link
Contributor

python3 setup.py install installs importlib-resources as a dep as expected ...

Installing via setup.py directly is deprecated. pip3 install [directory] seems to be the way forward.

You should see an error message like the following when you invoke setup.py directly:

SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.

Sorry for interjecting...

@vishwin
Copy link
Contributor

vishwin commented Dec 29, 2022

That's not an error, but a warning. And yes PEP-517 compliance is next on the list. Until then, setuptools can theoretically be pinned to 58, which is the last version with full setup.py support.

@vishwin
Copy link
Contributor

vishwin commented Dec 29, 2022

On second look, both of these commits are wrong. The importlib_resources package has to be used for Python < 3.10, not only for the new-style files() and as_files() API introduced in 3.9, but PEP-420 namespace package support only landed in time for 3.10. Both chirp.share and chirp.stock_configs are namespace packages, which are not to have __init__.py.

@kk7ds
Copy link
Owner

kk7ds commented Dec 29, 2022

On second look, both of these commits are wrong. The importlib_resources package has to be used for Python < 3.10, not only for the new-style files() and as_files() API introduced in 3.9, but PEP-420 namespace package support only landed in time for 3.10. Both chirp.share and chirp.stock_configs are namespace packages, which are not to have __init__.py.

Yep, this is exactly what I meant when I said "I thought that the thing we specifically needed was only in 3.10...". Further confirmed, I think, by the fact that it works fine for me in 3.9.

@asheplyakov
Copy link
Author

python3 setup.py install installs importlib-resources as a dep as expected ...

Yes. But it skips share and stock_configs (unless there is the __init__.py file in the directory)

Installing via setup.py directly is deprecated. pip3 install [directory] seems to be the way forward.

  1. pip calls setup.py install anyway, and prints the corresponding warning:
  removing build/bdist.linux-x86_64/wheel
  Building wheel for chirp (setup.py) ... done
  Created wheel for chirp: filename=chirp-py3dev-py3-none-any.whl size=1274863 sha256=1ab33209db1015ce3e02b9300a5daeef568b97d1d150da2ed31f8a7a7bf929f3
  Stored in directory: /usr/src/tmp/pip-ephem-wheel-cache-zqphcjaw/wheels/20/54/1a/b3128b303c56af26a8ebbd9f771961fb0e2c24b0734d4c6b09
  WARNING: Built wheel for chirp is invalid: Metadata 1.2 mandates PEP 440 version, but 'py3dev' is not
Failed to build chirp
Installing collected packages: chirp
[skipped]
  Running command Running setup.py install for chirp
  /usr/lib64/python3/site-packages/setuptools/dist.py:499: UserWarning: The version specified ('py3dev') is an invalid version, this may not work as expected with newer versions of setuptools, pip, and PyPI. Please see PEP 440 for more details.

  Running setup.py install for chirp ... done
  DEPRECATION: chirp was installed using the legacy 'setup.py install' method, because a wheel could not be built for it. A possible replacement is to fix the wheel build issue reported above. Discussion can be found at https://github.com/pypa/pip/issues/8368
Successfully installed chirp-py3dev

  1. I'm building an rpm package. python3 -m pip install is not the best tool for the task, so I use pyproject_installer instead like this
python3 -m pyproject_installer -v build
python3 -m pyproject_installer -v install --destdir=%buildroot

This one successfully builds a wheel and dist-info.

@vishwin
Copy link
Contributor

vishwin commented Dec 30, 2022

Yes. But it skips share and stock_configs (unless there is the __init__.py file in the directory)

If that was the case, I would not have had implemented this the way it is.

I'm building an rpm package

Any chance you have some outdated components in your chain by virtue of using RPMs provided by your operating system's repositories?

@asheplyakov
Copy link
Author

Yes. But it skips share and stock_configs (unless there is the __init__.py file in the directory)

If that was the case, I would not have had implemented this the way it is.

Apparently it didn't work as intended.

I'm building an rpm package

Any chance you have some outdated components in your chain by virtue of using RPMs provided by your operating system's repositories?

Which components should I check, and what are the correct versions? (the only versioned dependency in requirements.txt is wxPython>=4.0,<4.2.0 ; platform_system=="Linux", and I use wxPython 4.0.7)

@kk7ds
Copy link
Owner

kk7ds commented Dec 31, 2022

Yes. But it skips share and stock_configs (unless there is the __init__.py file in the directory)

If that was the case, I would not have had implemented this the way it is.

Apparently it didn't work as intended.

As noted, it does for me, on python 3.8, 3.9, and 3.10.

@kooscode
Copy link

kooscode commented Jan 5, 2023

I am trying to run chirp on Python3 to help test the drivers, but seems the installer doesnt work and is looking for python package wxPython which i cant install because somehow its looking for GTK2 ?

Any pointers on getting this running on Python 3.8 ?

@kk7ds
Copy link
Owner

kk7ds commented Jan 5, 2023

I am trying to run chirp on Python3 to help test the drivers, but seems the installer doesnt work and is looking for python package wxPython which i cant install because somehow its looking for GTK2 ?

Any pointers on getting this running on Python 3.8 ?

It would help to know what system you're running, but it works fine on python 3.8 without anything specific. See this page: https://chirp.danplanet.com/projects/chirp/wiki/DevelopersPython3Environment

Probably best to discuss further on the devel mailing list, separate from this PR.

@kooscode
Copy link

kooscode commented Jan 5, 2023

Ah - good point. sorry. I am running Ubuntu 20.04

@kooscode
Copy link

kooscode commented Jan 5, 2023

It would help to know what system you're running, but it works fine on python 3.8 without anything specific. See this page: https://chirp.danplanet.com/projects/chirp/wiki/DevelopersPython3Environment

Seems there is no branch py3 for this repo and the above instructions call out to checkout that branch?

fatal: 'origin/py3' is not a commit and a branch 'py3' cannot be created from it

@vishwin
Copy link
Contributor

vishwin commented Jan 5, 2023

Seems there is no branch py3 for this repo and the above instructions call out to checkout that branch?

fatal: 'origin/py3' is not a commit and a branch 'py3' cannot be created from it

Because py3 is now the main branch.

@goldstar611
Copy link
Contributor

After a very lengthy build process, I was able to get CHIRP and all of it's dependencies installed on a fresh Ubuntu 20.04 VM using the commands below. (It is probably not necessary to run pip as root and you'll see a message discouraging it.)

sudo apt install python3-pip libgtk-3-dev libpng-dev libtiff-dev libjpeg-dev libcurl4-openssl-dev
sudo -H pip3 install attrdict
sudo -H pip3 install "chirp[wx] @ git+https://github.com/kk7ds/chirp.git"

@kk7ds
Copy link
Owner

kk7ds commented Jan 5, 2023

Guys, on linux, don't install wxpython from pip - it builds the entire thing for no reason. Follow the instructions on the build page, install the distro package and then do the pip stuff. Should be two seconds.

@goldstar611
Copy link
Contributor

goldstar611 commented Jan 5, 2023

Ah, I knew there should be a faster way. Thanks for the tip Dan. And now I understand why wx is listed as an extra in setup.py

From https://chirp.danplanet.com/projects/chirp/wiki/DevelopersPython3Environment:

sudo apt install git python3-wxgtk4.0 python3-serial python3-six python3-future python3-requests python3-pip

then

pip install git+https://github.com/kk7ds/chirp

Another confirm that the current tip of master installs (and runs) on Ubuntu 20.04 with python 3.8.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants